Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Close
Copied to clipboard
Close
Unable to share or copy to clipboard
Close
🤖 Web Crawling Politeness
robots.txt, Rate Limiting, Crawl Delay, User Agents
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
32382
posts in
13.5
ms
local.html
: Social Discovery by Browser-Based
Crawling
lofihi.fi
·
1d
·
Discuss:
Hacker News
🕷️
Web Crawling
Stack
Overflow
for AI Agents
agentthreads.dev
·
1h
·
Discuss:
Hacker News
💻
Coding Agents
A better way to
crawl
websites with
PHP
freek.dev
·
20h
🕷️
Web Crawling
BrowseSafe
: Understanding and
Preventing
Prompt Injection Within AI Browser Agents
research.perplexity.ai
·
15h
🛡️
AI Security
Ask don't tell:
Reducing
sycophancy
in large language models
arxiv.org
·
1d
🎯
Colbert
Cloudflare’s new
Markdown
support shows how the web is
evolving
for AI agents
thenewstack.io
·
1d
🕷️
Web Crawling
The AI
Scraping
War: LLM
Crawlers
Are Breaking the Web
webdecoy.com
·
1d
·
Discuss:
r/webdev
💳
Content Monetization
New comment by
jedberg
in "Show HN: I built a sub-500ms
latency
voice agent from scratch"
news.ycombinator.com
·
11h
·
Discuss:
Hacker News
🎯
Colbert
Speed of
Sites
Matters
techrights.org
·
1d
🚀
Web Performance
An
AI-enhanced
web
reader
that does not get in your way
gistreader.com
·
2h
📰
RSS Reading Practices
The
Complicators
, The Drama Aggregators, and The
Avoiders
randsinrepose.com
·
21h
📰
RSS Reading Practices
Context
Engineering vs
Prompt
Engineering
newsletter.systemdesign.one
·
3h
·
Discuss:
r/programming
🪄
Prompt Engineering
Zero-Waste Agentic RAG: Designing
Caching
Architectures to
Minimize
Latency and LLM Costs at Scale
towardsdatascience.com
·
1d
💾
Prompt Caching
How Google’s Universal Commerce Protocol changes
ecommerce
SEO
searchengineland.com
·
22h
💰
Search Economics
OpenClaw
:
Setting
Up Your First Personal AI Agent
kill-the-newsletter.com
·
20h
🛡️
Open Policy Agent
Architecting
and
Evaluating
an AI-First Search API
research.perplexity.ai
·
15h
🕷️
Web Crawling
Parse
, Don't
Guess
architecture-weekly.com
·
19h
💾
Binary Formats
How fast is fast enough?
Rethinking
web performance in
pragmagical
terms
speedcurve.com
·
1d
🚀
Web Performance
Can LLM
chat
be less
prolix
?
lesswrong.com
·
16h
💾
Prompt Caching
Blog -
Measuring
Web Accessibility from
Crawl
Archives
commoncrawl.org
·
1d
🕷️
Web Crawling
Loading...
Loading more...
Page 2 »
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help